1. Field of the Invention
The present invention relates to an image processing apparatus in an image output system, a printer system, an image display system, etc., having a function of automatically correcting the gradation of color image data, such a technology being able to be applied to a digital camera, a scanner, etc.
2. Description of the Related Art
In recent years, the opportunity to treat a photograph image as digitized data is increasing upon spread of a personal computer, Internet, a home printer, and also, enlargement of the storage capacity of a hard disk drive, etc. Generally in connection therewith, a digital camera, a film scanner, etc. are used as means for inputting digital photograph image data (referred simply as image data, hereinafter).
An automatic exposure control device for always keeping the exposure at a time of photography the optimum is provided in a digital camera. Although there are various methods in an exposure control type, a method for dividing a screen into a plurality of areas for light amount detection, applying an weight for each area, and taking a weighted average thereof, and, then, based thereon, adjusting an iris diaphragm, shutter speed, etc. of the camera is commonly used.
However, in such an exposure control method, a problem may occur. Especially, in a backlight state in which a light source exists behind a main photographic subject where there is a large luminance difference between the background and the main photographic subject, the exposure may be corrected in minus direction by an automatic exposure control function as a result of being influenced by the background brightness when the exposure does not suit the photographic subject, and, as a result, the photographic subject becomes dark in a resulting picked-up image.
Further, in a photography in the night by using a stroboscope light/flash light, since it is the requisite that stroboscope light shines upon a photographic subject, an iris diaphragm and shutter speed of the camera are fixed to defaults. As a result, if the distance between the stroboscope and the photographic subject is too far, the light does not arrive at the subject, and, thus, the photographic subject becomes dark in a resulting picked-up image also in this case.
There, the exposure correction function may be provided for adjusting the brightness of a photographic subject which has unsuitable brightness with respect to the scene to the brightness suitable to the scene. For example, a photographic subject having become generally dark due to underexposure, or a photographic subject having become dark due to backlight may be made brighter thereby. Alternatively, a photographic subject on the overexposure may be made darker thereby.
The exposure correction in a camera employs a common method for altering an iris diaphragm and a shutter speed of the camera, in order to adjust receiving light amount in a lens. Moreover, in a case of a printer or a display device, the exposure correction function for optimizing the brightness of an output signal with respect to the brightness of an input signal etc. using an input-to-output conversion/transformation function (gradation correcting curve) etc. may be provided.
Since correcting the exposure so as to adjust the brightness of image data described above and the gradation correction processing according to the present invention have the same purpose, they are both referred to as gradation correction processing, hereinafter.
Various methods have been proposed for automatically performing gradation correction of image data for coping with unsuitable exposure control such as those described above performed in image pickup occasions.
For example, according to an image processing method, a main image feature quantity of an original image is calculated by reading the whole original image, a predetermined density range of a photographic subject is calculated from the calculated main image feature quantity, and only the pixels falling within the above-mentioned predetermined density range are extracted from the whole original image, and an original image feature quantity is determined therefrom. Then, from the thus-obtained original image feature quantity, an exposure amount is determined. Thereby, the exposure correction is performed appropriately on an image such as backlight image in which a density difference between the background and photographic subject is large (see Japanese laid-open patent application No. 7-301867).
Moreover, according to another image processing method, a conversion curve through which image data conversion is made so that saturation does not take place throughout the image is created, and therethrough, backlight correction is performed (see Japanese laid-open patent application No. 10-79885).
An image processing method disclosed in Japanese laid-open patent application No. 10-79885 sets a refraction instruction value at a low luminance portion in an input image. Then, as an output value for the refraction instruction value, a value “220” which is somewhat smaller than the maximum luminance value “255” is selected. Then, a conversion table is created in which this point is regarded as a refraction point, and until an input luminance value reaches the refraction point, the relationship between the input and output is linear along a straight line between the original point and the refraction point. Then, after that, the relationship between the input and output is linear along a straight line between the refraction point and the input/output maximum coordinate point (255, 255). Then, by using the created conversion table, an input luminance is converted and output. Thereby, a shadow part occurring in a backlight image is made brighter, while gradation crush in a brighter part is avoided. Gradation crush means that originally different gradation levels become the same level, hereinafter.
According to Japanese laid-open patent application No. 2000-134467, an image data which shows a relevant image is input, a histogram on this image is created, and, based on this histogram, it is determined whether or not the relevant image is a backlight image. Then, according to the determination result, image processing conditions are set up, and, thus, it is possible to perform image processing suitable to the backlight scene.
However, it is difficult to appropriately correct the brightness on a photographic subject so that satisfactory gradation expression is ensured while excessive correction is avoided. When a correction amount is derived only from the value on the photographic subject as in the art disclosed in Japanese laid-open patent application No. 7-301867, gradation crush may occur in background, gradation crush or hue displacement/deviation may occur on high chroma color.
Further, in case a gradation correction curve is created as in Japanese laid-open patent application No. 10-79885, and color processing is performed according to the gradation correction curve, the luminance component is avoided from being saturated, a color component may be saturated and it may cause gradation crush or hue saturation. Moreover, when the gradation correction curve is set up so that saturation be avoided from occurring throughout the image, the correction may not function effectively.
Since the brightness differs between a photographic subject and a background extremely in the above-mentioned night photography or backlight photography, the photographic subject may be extracted from the image, and the exposure may be determined therefrom, as mentioned above.
In this regard, various technologies of extracting the photographic subject have been proposed. For example, according to one method, a portion in which a flesh color occur many times is extracted from an image, i.e., extraction is made mainly for a face of human being. According to another method, a portion having a large edge amount is determined as a focused area, and, thus, is determined as a photographic subject.
However, it is difficult to extract an area having a fixed color such as flesh color from an image obtained through unsuitable exposure control. Further, since a focusing is made even in detail of an image picked up by a digital camera depending on the size of image-pickup device, the edge amount becomes larger in background in a backlight image in which a photographic subject is darker than the background and also there is little contrast, and, thus, extraction preciseness for the photographic subject is not satisfactory.
On the other hand, Japanese laid-open patent application No. 5-236340, for example, discloses a method of removing the background, instead of extracting the photographic subject. In this method, comparison is made on luminance and hue of arbitrary pixel with known average luminance and average hue of background image, and, in case the different therebetween is larger than a predetermined threshold, the pixel is regarded as a pixel of the photographic subject. In this method the threshold is changed according to the average luminance and average hue of the background image.
However, according to this method, it is necessary that information on the background is previously known. Furthermore, problematically the obtained average hue becomes a hue different from any hues included in the background in case various colors are included there evenly. If so, this information cannot be used as a proper extraction determination reference.
According to Japanese laid-open patent application No. 5-91324, an image is divided into a plurality of blocks/divisions, density information is obtained every block, and, thereby, background blocks are extracted. Thus, according to this method, a density average value DM on respective pixels, a density average value Dn on each block, and an average scattering value Sn thereof are obtained. Then, densities of peripheral blocks and density continuity are determined from Dn and Sn. Thus, the background is determined.
However, according to this method, as the background is extracted by using density information, image parts having completely different colors may be extracted as the background in case they have same density. Accordingly, it may not be possible to perform highly precise determination (background extraction determination).